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separately converting features within the zones into vector-based caricature 
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ify the caricature to reflect a: particular selected cutistic style (142), such 
as a style reminiscent of popular comic book, television or motion picture 
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INTERNET-BASED METHOD A^a^ APPARATUS FOR GENERA _ . . 

CARICATURES 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

5 The invention generally relates to e-commerce web sites and in particular to an e-commerce 

web site for automatically converting an input image, such as a user photograph, into a line art 
caricature or other caricature. 

2. Description of Related Art 

Millions of e-mail messages or other computer mail messages are transmitted via the 

10 Internet every day. As the speed and bandwidth of the Internet has increased, and as the speed of 
modems used by individual users has increased, it is becoming increasingly feasible to send a 
photograph of the sender along with the computer message to thereby personalize the message. 
This is particularly desirable for holiday greetings, birthday greetings and the like. Likewise, it 
has become feasible for participants of chat rooms, multi-user computer games, and the like, to 

15 employ visual Avatars for use in representing themselves within the chat rooms or computer 
games. Accordingly, some Internet users now scan a photograph of themselves into their computer 
to create a bit-mapped image for forwarding along with computer messages or for use as an 
Avatar. Often, however, users are unsatisfied with the bit-mapped photographic image, and would 
prefer to modify the image as needed to generate an image with which they are more comfortable. 

20 In other cases, users might want to provide an amusing caricature of themselves, rather than a 
simple bit-mapped photograph. To this end, a user may wish to convert his or her picture into a 
cartoon caricature having a particular artistic style, perhaps one reminiscent of popular comic 
book, television, or motion picture animation styles. Also, bit-mapped images are typically fairly 
large images requiring considerable resources to upload and download via the Internet. 

25 Accordingly, many users might prefer to convert their bit-mapped photograph into amore compact 
image format, such as vector-based format, before sending via the Internet. 

Accordingly, it would be desirable to provide an Internet web site for permitting 
users to easily convert bit-mapped photographs into vector-based caricature images, to selectively 
enhance or modify aspects of the caricature images, or to convert the caricature into a cartoon 

30 having a selected artistic style. HeretoforCi as far as the inventors are aware, no Internet web sites 

1 
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are available providing this combination of features. Moreover, an attempt to create such a web 
site using* conventional automated image processing techniques may not be feasible. 
Conventional techniques for converting a bit-mapped photograph into a vector-based image 
typically process the entire input photograph to extract the vectors, often resulting in an output 
5 image that emphasizes shadows or other artifacts in the photograph unrelated to the actual 
features of the object of the photograph, such as the eyes, nose and mouth of a person pictured 
in a photograph. Hence, such conventional techniques may not be suitable for use in a web site 
for converting an input photograph into a caricature image. Conventional techniques for 
enhancing or modifying aspects of a photograph also typically process the entire input image 

10 resulting in similar problems. For example, one technique that may be employed for modifying 
aspects of an image is to morph the image together with another image. However, typical 
morphing techniques are applied uniformly throughout the images, resulting in a final image 
wherein distinctive facial features, such as those of the eyes, nose, and mouth, are often 
considerably distorted. In this regard, portions of the eyes of one image may be morphed with 

15 portions of the nose of another image resulting in a highly distorted final image. These problems 
are even more significant when morphing a photograph with a cartoon character, typically 
resulting in a highly comical but rather unsatisfactory final image. Moreover, conventional 
automated morphing techniques typically require considerable processing power and hence may 
be relatively slow or expensive to perform, particularly for use within a heavily accessed Internet 

20 site. Conventional morphing techniques can be fairly complex and many Internet users may not 
wish to learn techniques needed to control a potentially complex morphing process. Hence, such 
conventional techniques may not be suitable for use in a web site to selectively enhance or 
modify aspects of caricature images or to convert caricatures into cartoons. Also, conventional 
morphing techniques typically require that one input image be morphed with another image and 

25 hence may not be suitable for generating a cartoon caricature merely having a certain selected 
artistic style. 

Furthermore, as the speed and bandwidth of the Internet increases, it becomes 
increasingly feasible to employ animation in connection with computer messages, chat room 
Avatars, and the like. Problems associated with conventional techniques for generating and 
30 processing photographs. Avatars, and the like are even more significant in circumstances wherein 
the images need to be animated. 

Accordingly, it is desirable to provide an Internet web site for permitting users to easily 
convert bit-mapped photographs or other images into vector-based caricature images, to 
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selectively enhance or modify aspects* of the caricature images, or to convert the caricatures into 
cartoon caricatures having a selected artistic style. It is also desirable to provide improved image 
processing techniques for use with such a web site. In particular, it would be desirable to provide 
a technique for easily and efficiently converting a bit-mapped image into a caricature image, 
5 which retains distinctive facial features of the bit-mapped image. It also would be desirable to 
provide a technique for efficiently converting the caricature into a cartoon which does not result 
in a highly distorted final image and which permits control of the style of animation. It also 
would be desirable to provide a technique for efficiently combining a pair of caricature images 
together in a manner that does not result in a highly distorted final image. It is to these ends that 
10 aspects of the present invention are primarily directed. 

SUMMARY OF THE INVENTION 
In accordance with a first aspect of the invention, an interactive network web site is 
provided that includes an image selection unit for receiving a user selection of an image and a 
caricature generation unit for automatically generating a caricature image representative of the 

15 selected image. The term caricature as used herein is not intended to imply that the caricature 
is necessarily an exaggerated or distorted representation of the selected image. For example, the 
caricature may be a realistic line art representation of the selected image. 

In an exemplary embodiment of the first aspect of the invention, the interactive network 
web site is an Internet site. The selected image is a bit-mapped photograph either received via 

20 the Internet or selected from a database of stored images. The selected image is converted to a 
vector-based caricature for embedding within a computer message, such as a holiday or birthday 
greeting. The caricature may also be used as an Avatar within chat rooms or multi-player 
computer games or for other purposes. By using a caricature, rather than a simple bit-mapped 
photograph, the image of the user is therefore far more distinctive than one merely containing 

25 an ordinary photograph. The use of a caricature is particularly advantageous for those who do 
not wish to use a photograph of themselves, perhaps because the photograph is unflattering or 
because the user is otherwise uncomfortable with transmitting a photograph, unmodified, via the 
Internet. Also, a vector-based image typically requires fewer bytes than a bit-mapped photograph 
and hence can be easier to transmit, store and display via the Internet, 

30 Also, in the exemplary embodiment of the first aspect of the invention, the caricature, 

once generated, can be modified to adopt a distinctive artistic style drawn from pre-stored artistic 
styles, such as styles based on popular television or motion picture cartoon characters or original 
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artistic styles, or can be merged with another caricature. In this manner, the user can generate 
a more amusing and memorable caricature for use via the Internet. 

In accordance with a second aspect of the invention, a method is provided for use by a 
computing system to convert an input image into a computer representation of a caricature image. 
5 Initially, a set of image zones is automatically located within the input image. Portions of the 
input image lying within each image zone are separately converted into individual respective line 
representations. The individual respective caricature representations are then combined to yield 
a combine caricature representation. 

In an exemplary embodiment of the second aspect of the invention, the input image is a 

10 bit-mapped photograph of a face. The set of image zones automatically located within the input 
image are facial image zones such as the left eye, the right eye, the nose and the mouth. By first 
locating image zones within the bit-mapped photograph, then separately converting portions of 
the input image lying within the image zones into caricature representations, the resulting 
combined caricature representation can more effectively retain the distinctive features of the 

15 photograph than is achieved using conversion techniques of the type which uniformly process 
the entire input image. Hence, a more satisfactory caricature image may be generated. 
Preferably, separate conversion algorithms are used within the separate image zones, each 
specifically adapted for processing the distinctive features of the respective image zone. Also, 
preferably, the resulting caricature representation is defined by geometrical nodes associated with 

20 each image zone, which collectively define the features within the image zone. Hence, the 
resulting image representation is highly compact and can be easily processed, perhaps to generate 
multiple caricatures each adopting a different distinctive artistic style. 

In accordance with a third aspect of the invention, a method is provided for use by a 
computing system to render a caricature image adopting a selected artistic style. In accordance 

25 with the method, a caricature image representation is input and an artistic style is selected. Then, 
an image is rendered based on the caricature image representation while applying the selected 
artistic style. 

In an exemplary embodiment of the third aspect of the invention, the caricature image 
representation is a vector-based caricature representation generated from a user image or other 
30 image. The selected artistic style is representative of popular television or motion picture cartoon 
characters or an original artistic style. The image rendered by the computing system based on 
the vector-based caricature and the selected artistic style is a cartoon caricature. The artistic style 
defines general features for use in generating the cartoon caricature, such as whether the eyes. 
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nose and mouth of the caricature should be large or small. The artistic style also defines colors 
or line drawing textures for use in rendering the caricature, such as whether lines should be bold 
and curved or thin, rough and angular. By rendering a caricature so as to adopt a selected artistic 
style, a memorable line caricature is thereby created for use via the Internet or for other 
5 applications. 

In accordance with a fourth aspect of the invention, a method is provided for use by a 
computing system to combine computer representations of first and second images. Initially, a 
set of corresponding image zones within the first and second images are located. The image 
zones of the first image are separately modified based on characteristics of corresponding image 
10 zones of the siecond image to yield separately modified image zones. Then, the separately 
modified image zones are combined to yield a computer representation of a modified image. 

System embodiments are also provided. Other objects, features and advantages of the 
invention will be apparent from the descriptions provided below in combination with the 
accompanying drawings. 

1 5 BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 illustrates an Internet-based method for generating a computer message having 
a caricature incorporated therein. 

FIG. 2 illustrates a method for generating a caricature based on an input bit-mapped 
photograph for use within the method of FIG. 1. 
20 FIG. 3 illustrates a method of modifying the caricature generated by the method of FIG. 

2 based on a pre-stored artistic style also for use within the Internet-based method of FIG. 1. 

FIG. 4 graphically illustrates the generation of a caricature based on a bit-mapped 
photograph performed by the method of FIG. 2 and further graphically illustrates modification 
of the caricature based on an artistic style as performed by the method of FIG. 3. 
25 FIG. 5 illustrates a method of merging a caricature generated by the method of FIG. 2 

with another caricature also for use within the Internet-based method of FIG. 1. 
FIG. 6 illustrates a system for performing the methods of FIGS. 1 - 5. 



5 
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DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 

With reference to the figures, preferred and exemplary embodiments of the invention will 
now be described. The techniques of the invention will primarily be described with reference 
to an Internet-based method for generating a computer message having a line art caricature or 
other caricature embedded therein. The caricature is generated by converting a photograph into 
a caricature image. Then, depending upon the programming of the system and the commands 
entered by a user, the caricature may be modified so as to adopt a selected artistic style or merged 
with another caricature. However, principles of the invention are applicable for use in other 
systems and to achieve other ends. The techniques are not Hmited for use with interactive web 
sites but may be employed in connection with stand-alone computers or computer kiosks. The 
techniques described herein for converting a photograph into a caricature need not be performed 
in conjunction with generating a computer message'. The techniques for modifying a caricature 
so as to adopt a selected artistic style need not be performed in conjunction with images derived 
from original bit-mapped photographs. Accordingly, it should be understood that the examples 
described herein are provided for illustrative purposes only and are not to be taken as limiting 
the scope of the invention as set forth in the accompanying claims. 

Initially, the Internet-based method for generating a computer message having a caricature 
embedded therein will be described with reference to FIG. 1. Techniques for converting a bit- 
mapped photograph into a caricature will be described with reference to FIGS. 2 and 4. 
Techniques for modifying the caricature to adopt a selected artistic style will be described with 
reference to FIG. 3 and 4. Techniques for merging the caricature with another caricature will 
be described with reference to FIG. 5. Then a system performing the techniques of FIGS. 1 - 
5 will briefly be summarized with reference to FIG. 6. 

The method of FIG. 1 is performed by an interactive network web site computer system 
connected to the Internet and accessible as an Internet web site by Internet users via a web 
browser. Initially, at step 100, the user selects a general format for the computer message from 
a group of predetermined message formats stored within the computer system. Each computer 
message format includes space for a picture, such as a picture of a person, couple, house, pet, 
boat, car, skyline, horizon, landscape etc. Thousands of individual computer formats are 
preferably provided, categorized according to the general purpose of the computer message. 
Examples of general categories include holiday greetings, birthday greetings, reminders, 
anniversary greetings, vacation greetings, and wedding invitations. Within each general category. 
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numerous specific formats are provided corresponding to the category. For example, for 
holidays, separate formats are provided for Christmas, Easter, Chanukah, Fourth of July, etc. For 
birthday greetings, specific formats are provided for different age groups as well as for different 
genders. Once the user has selected a format, then, at step 102, the user inputs or selects specific 
5 text or sound recordings for incorporation into the format. For example, the user may modify 
a prewritten birthday greeting to personalize the greeting or may provide entirely new text to 
replace prewritten greetings. The user may recite a message into a microphone of the user's 
computer, which digitizes the voice, sends it to the web site computer for incorporation within 
the message. At step 104, the computer system embeds the text or voice recording, if any, 

10 provided by the user into the computer message format. 

As noted, each message format includes space for a picture. At step 106, the system 
determines the subject mater of picture and the particular camera angle required for the picture 
to be inserted into the message, based upon the pre-stored format of the message. For example, 
for a message format with space for a facial photograph, the system determines whether a front 

15 view, side view, left angle, right etc. is required, then prompts the user to input a bit-mapped 
photograph having the prescribed camera angle. The user inputs the photograph at step 108 by 
transmitting the image via the Internet from a computer of the user to the web site computer 
system. The bit-mapped photograph is initially generated using a digital camera or other digital 
input device or by scanning a photograph using a digital scanner or by uploading or downloading 

20 an image from a computer database. For example, if the user's computer has a digital camera 
mounted thereto, the system may prompt the user to take a digital photograph using the digital 
camera for immediate transmission via the Internet. Alternatively, the user selects from among 
photographs previously input into the system and stored therein or from among previously 
converted vector-based images. To this end, each individual user may initially download 

25 numerous photographs into the computer system for storage therein to provide a library of 
images. The library may include various pictures taken at different angles of the user, his or her 
friends or fanwly, pets, houses, boats etc. In any case, once the user has input a bit-mapped 
photograph or selected from among pre-stored photographs then, at step 110, the system 
generates a caricature of the photograph and displays the caricature for the user by transmitting 

30 the caricature via the Internet to the user's computer. The caricature is a vector-based image. 
Geometric nodes defining the vectors are stored by the system. Depending upon the 
programming of the computer system and upon any conmiands entered by the user, the 
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generation of the caricature is performed entirely automatically or is performed in conjunction 
with specific user commands. At step 112, subject to the control of the user, the caricature may 
be modified to adopt the characteristics of a selected artistic style to yield a cartoon caricature 
or may be merged with another caricature. At step 114, the user has the opportunity to review 
5 the caricature that has been generated and to accept or refuse the caricature. Assuming that the 
user is satisfied with the caricature, then the system embeds the caricature within the selected 
computer message format, at step 116, and then sends the computer message at step 118. 

Thus, FIG. 1 illustrates an Internet-based technique for embedding caricatures within 
computer messages, particularly within holiday greetings, birthday greetings, and the like. In this 

10 manner, the user may personalize the greeting to a much greater extent than can be achieved 
merely by incorporating text greetings. Moreover, by generating and sending a caricature, rather 
than a simple bit-mapped photograph, the message received by the recipient is likely to be far 
more distinctive than one merely containing a simple bit-mapped photograph. The creation of 
a caricature is particularly advantageous for those who do not wish to send a simple photograph, 

15 perhaps because the photograph is unflattering or because they are simply uncomfortable with 
transmitting their photograph via the Internet. Also, as will be more fully explained below, in 
generating the caricature, the user has the opportunity to adjust or modify features to enhance the 
visual representation as desired. 

Although described with reference to a system which first converts bit-mapped images 

20 into caricatures then modifies the caricature based on a selected artistic style, the general 
techniques of FIG, 1 alternatively may be performed to just create an initial caricature, or to just 
modify a caricature based on an artistic style, or to just merge a caricature image with another. 
Also, the system may be configured to convert among different types of image formats, such as 
from vector-based to vector-based rather than only from bitmapped to vector-based. The final 

25 caricatures or other final images need not be employed only in conjunction with computer 
messages, but may be employed as Avatars for use in chat rooms, multi -player computer games, 
and the like or may merely be printed out and not used via the Internet at all. Also, the final 
caricature for embedding in the message may be animated. Or the final caricature need not be 
embedded within a message at all. As will be described more fully below, the caricatures are 

30 internally represented as a set of geometrical nodes that facilitate animation. For the purposes of 
animation, the input image may be an entire sequence of images of a person, object or animal in 
motion, with each image converted into a caricature to achieve an animated sequence. The input 
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image may comprise motion capture data from which animation can be generated. Depending 
upon the implementation, the system may incoipora:te the animated caricature within a message 
story vignette, perhaps to show the caricature blowing out the candles of a birthday cake. Also, 
3-D caricatures may be generated by, for example, processing multiple views of the same person 
5 taken from different angles. Furthermore, the computer network of FIG. 1 need not be the 
Internet, but may be an Intranet or other limited computer network. Moreover, when 
implemented in connection with the Internet, many of the steps of the invention may be 
performed within the user's computer rather than within the web site server, perhaps using plug- 
ins or the like. Also, aspects of the method need not be implemented via a computer network at 

10 all, but instead may be implemented within a stand-alone computer or kiosk. To this end, 
software performing the steps of the method may be stored on CD-ROM's, floppy disks or the 
like for loading into user computers. Also the method may be implemented via a channel rather 
than a web site. As can be appreciated, numerous other modifications or applications of the 
general techniques of FIG. 1 may be provided consistent with the general principles of the 

15 invention. 

Techniques for converting a bit-mapped image into a vector-based caricature will now 
be described for an example involving a facial image with reference to FIG. 2 in conjunction 
with FIG. 4. The steps of FIG. 2 may be modified as needed to process other images, such as 
houses, cars, boats, etc. Initially, at step 120, the system verifies the integrity of the bit-mapped 

20 image (item 121 of FIG. 4) input by the user. To this end, the system verifies that the exposure, 
contrast, focus and other image quality features are all within acceptable pre-detemiined ranges. 
If the image, fails the integrity test, the user is prompted to input an alternative image or to 
otherwise correct the image. The system preferably incorporates various tools for altering the 
image, such as changing the contrast and the like, so that the user may attempt to remedy any 

25 defects in the image. Alternatively, the system may automatically perform steps directed to 
remedying the image defects on behalf of the user, then present the corrected image to the user 
for verification that the image is acceptable. 

At step 122, the system then analyzes the bit-mapped image to extract the outline of the 
face (item 123 of FIG. 4) contained within the image. This is performed by analyzing the 

30 differences between two or more adjacent pixels, such as differences in brightness, contrast, color 
and the like. One exemplary technique that may be employed by the system is to process the bit- 
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mapped image using the following filter applied to each block of nine adjacent bits: 
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5 The aforementioned filter operates to highlight changes in contrast within the image. Any 

part of the image where an absolute filtered value is higher than a predetermined threshold is 
defined as an "edge" of the image. Additional information regarding edge detection may be 
found within a chapter entitled Noise Thresholding in Edge Images by John Schlag, (pp. 105- 
106) of Graphics Gems edited by Andrew S, Glassner, Academic Press (1990). 

10 Having identified the edge of the input image, the system next locates predetermined 

image zones such as the eyes, nose, mouth, eyebrows, forehead, hair profile, hairline, jaw line, 
chin, cheekbones, and ears within the image by applying a facial image zone template to the 
oudine. (For other image types, such as houses or boats, other image zones are defined.) More 
specifically, within step 124, the system inputs an image zone template (item 125 of FIG. 4) 

15 having predetermined image zones. In the example, of FIG. 4, only a few of the image zones 
are illustrated. At step 126, the system adjusts the size and location of the image zone template 
to fit the outline of the bit-mapped image. This may be achieved by fitting an face perimeter 
template, such as an oval or round template, to the of the image while simultaneously adjusting 
the sizes and locations of intemal image zones in accordance with the adjusted size and location 

20 of the oval face perimeter template. This may be performed by minimizing an average distance 
between points on the oval perimeter and points within the outiine derived from the bit-mapped 
image. When fitted properly, the individual image zones of the template fit over the 
corresponding facial features in the bit-mapped image. In other words, the right eye zone 
template fits over the right eye of the image; the mouth zone template fits over the mouth of the 

25 image, and so on. To ensure proper fit, the technique employed to fit the oval perimeter to the 
face is preferably performed so as to be relatively unaffected by any portions of the outline that 
may represent beard or hair, rather than facial outline. To this end, left and right portions of the 
oval template are more closely fitted to left and right portions of the bit-mapped image outline 
as compared with the top and bottom portions of the oval perimeter and top and bottom portions 

30 of the image outline. Also, the forehead zone may be independently fitted first, with the other 

10 
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facial zones thereafter fitted below the hairline. Note that the operations of step 126 are 
performed internally within the computer system using digitized data not requiring actual super- 
positioning of a graphic template over the facial image. The graphic template illustrated in FIG. 
4 is provided to symbolically illustrate the operations performed by the computer and not to 
5 imply that a graphic template is required. 

Thus, the system automatically identifies the locations of pre-determined features of the 
input image without requiring further user input. This is a significant advantage over systems 
that would require the user to separately identify facial features, which may be a laborious 
process. However, for those users who wish to assist the computer system in identifying the 

10 facial features, the system may be progranuned to display the bit-mapped image along with a 
graphical image zone template and permit the user to manually superimpose one above the other. 

Having identified the location of the various image zones within the bit-mapped image, 
the system then, at step 128, independently processes portions of the bit-mapped image lying 
within each respective image zone to generate a vector-based representation of features within 

15 the zone. To this end, the system analyzes the bit-mapped image lying within each respective 
image zone and identifies pre-determined geometric nodes therein. The nodes include specific 
geometrical points, tangents and the like. Within item 131 of FIG. 4, five exemplary nodes (A) - 
(E) are identified within the bit-mapped image of the right eye. As can be seen, the nodes 
correspond to the left and right comers of the eye, the center of the pupil, and horizontal tangents 

20 lying along the top and bottom portions of the eye. To provide a more detailed representation 
of the eye, a greater number of nodes may be employed. (The eyebrow of item 131 is analyzed 
separately from the eye; resulting in separate nodes, not specifically show in FIG. 4.) Also, in 
step 128 the bits defining the edge of the face are processed to generate a vector representation 
of the outline of the face. 

25 The characteristics of particular nodes for identification within each zone within step 128 

are pre-determined and are stored within the system. Hence, during step 128, the system need 
only locate the position or other characteristic of each predetermined node within the image zone 
based on the identifying characteristics of the nodes. The nodes, once identified, are then saved 
by the system for use in generating a caricature. A caricature of the eye, for example, can be 

30 subsequently generated by drawing an ellipse connecting the left and right comer nodes and the 
upper and lower tangent nodes and by drawing a pupil having the size and location identified by 
the pupil node. In this manner, the original bit-mapped image is converted to a compact set of 

11 
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geometrical nodes from which the caricature can be generated subsequently. An exemplary 

caricature is shown as item 135 of FIG. 4, 

As noted, the conversion of the bit-mapped representation to a vector-based representation 
is performed separately and independently for the image zones using separate conversion 
techniques appropriate to the features within those zones. For example, the conversion algorithm 
for identifying the geometrical nodes within the right eye is optimized for identifying distinctive 
features of the eye, such as the left and right comers, upper and lower eyelid tangents and the size 
and center of the pupil. The conversion algorithm for converting the bit-mapped image 
corresponding to the mouth into a caricature image of the mouth is optimized for identifying 
features of the mouth. 

For the example involving die eye, an exemplary conversion algorithm for use in locating 
the pupil of the eye is as follows: 



Above equation is used to find the pupil by finding the best circle fitting a set of points 
by minimizing the integral square error (E). Values Xj and represent the locations of bits 
corresponding to the pupil. Variable r represents the radius of the pupil and variables x^ and y^ 
represent the location of the center of the pupil. Variables r, x^, and y^ are determined by 
minimizing E over all of bits Xj and y^. In this manner, the size and position of the pupil is 
identified and the bit-mapped image of the pupil is thereby converted into a vector representation 
defined by solely by r and center location x^, and y,, which define node (e) representing the pupil. 
Hence, a graphic representation of the pupil thereafter may be generated merely by displaying 
a filled circle of radius r and center location x^, and y^. The same conversion algorithm may be 
applied for use with other round image zone features as well, such as automobile wheels within 
a picture of an automobile being converted into a caricature. 

To approximate the outline of the eye, or any other generally elliptical feature, the 
following equation may be employed to first determine the semi-major and semi -minor axis of 
the elliptical feature: 

12 
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The semi-major (a) and semi-minor (b) axes of the ellipse are determined for a given set 
of n points on the average where the coordinates and are calculated with respect to a system 
of coordinates having its center at the center of the ellipse, and the axes parallel to the ellipse 
5 axes. Once the semi-major and semi-minor axes are determined, other equations may be 
employed to determine the location of the center of the ellipse with respect to other features of 
the caricature. Then, the left and right and top and bottom points on the ellipse are determine for 
storage as nodes (a) - (d). In this manner, the size, shape and position of an elliptical facial 
feature are identified and the bit-mapped image of the feature is thereby converted into a vector 

10 representation. Hence, a graphic representation of the elliptical facial feature thereafter may be 
generated merely by displaying an ellipse of the determined size, shape and location. For the eye, 
as an alternative, other conversion algorithms maybe employed for directly locating tangents (b) 
and (d) and comers (a) and (c) which do not require first determining the semi-major and semi- 
minor axes. Also, more complex conversion algorithms may be employed to generate a greater 

15 number of nodes per facial feature so as to more precisely approximate the feature. 

Thus, within each image zone, one or more conversion algorithms are employed, with the 
particular conversion algorithms selected based on the features expected to be found in the image 
zone. In this manner, the overall bit-mapped image to caricature image conversion algorithm is 
optimized for each image zone. For an image zone expected to contain circular and elliptical 

20 features, the above conversion algorithms may be employed. For an image zone expected to 
contain curved lines, such an image zone covering the eyebrows, conversion algorithms adapted 
to locating curved lines are employed. For an image zone expected to contain square features, 
such image zone applied to the windows of a house, conversion algorithms adapted to locating 
squares are employed. Each conversion algorithm converts portions of the image into one or 

25 more geometrical nodes, which may be represented by numerical values such as object center 
locations, radii, lengths, tangent points, etc. The geometrical nodes are stored along with an 
indication of the type of node, i.e. whether the node defines a circle, a portion of an ellipse, a 
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Straight line etc. As can be appreciated, numerous different algorithms may be employed 

depending upon the subject matter of the image and the particular image zone contained therein. 

No attempt is made herein to describe or identify the many possible conversion algorithms. 

Those skilled in the art can readily develop conversion algorithms adapted for particular image 

5 types and image zones. General information pertaining to the generation of a vector-based 

representation may be found within Vectorization of Digital images Using Algebraic Curves, 

Zhang et ah. Computing and Graphics, Vol. 22, No. 1, pp. 99-101 (1998). 

Following the steps of FIG. 2, the caricature is generated by drawing an image using all 

the geometric nodes identified in step 128 with all the image zones (item 135 of FIG. 4) 

10 corresponding to the original input image. When generating the caricature, the system preferably 
applies different textures to different image features. For example, when drawing the eyebrow, 
the system applies a predetermined rough texture. Hence, although the eyebrow may be 
internally represented using only a set of nodes defining a simple geometrical arc, the eyebrow 
drawn in the caricature is a more realistic eyebrow. In general, every feature of the image as 

15 represented by the geometrical nodes may be processed using textures when generating the 
caricature. When generating the caricature, the system also preferably applies different colors 
to the different image features. The colors may be determined by sampling corresponding 
portions of the input image. As one example, the pupil may be sampled to determine the color 
of the eye, then that color is employed when generating the pupil in the caricature. Alternatively, 

20 rather than applying colors and textures, a simple line art caricature can be generated. 

The caricature is displayed to the user to verify that the character is acceptable to the user. 
If acceptable, the caricature may thereafter be embedded into the aforementioned computer 
message or perhaps employed as an Avatar for use within a chat room, multi-player computer 
game or the like. If the caricature is unacceptable, the user may generate a new caricature based 

25 upon a newly input photograph or, depending upon the programming of the system, may adjust 
various parameters used in generating the caricature in an attempt to provide a more acceptable 
caricature. For example, the system may provide tools to permits the user to manually control 
the identification of the location of the eyes, nose and mouth if it appears that the automatic 
determination performed by the system is not accurate. Preferably, also, the user is provided with 

30 tools for controlling the textures use in generating the caricature from the geometrical nodes so 
as to provide greater control over the resulting caricature. Also, the user is preferably provided 
with tools to permit the caricature to be modified so as to incorporate a wig, hat, etc, or if the 
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caricature only covers the face, to superimpose the face on a cartoon body or within a cartoon 
vehicle. Other tools are provided to permit the user to animate the caricature, perhaps to make 
the eyes blink or the mouth move. Since the caricature is internally represented by nodes, it is 
easy to animate the caricature by sequentially rendering the caricature while changing node 
locations. For example, an eye can be made to blink by alternately changing the spacing between 
nodes (b) and (d) of item 131 of FIG- 4. As another example, the mouth can be made to open 
and close by alternately changing the spacing between nodes define top and bottom portions of 
the lips. If a voice recording is provided for use with the caricature, the movement of the mouth 
may be synchronized with the voice to provide the illusion that the caricature is speaking the 
words of the voice recording. 

Thus, FIG. 2 illustrates a method for converting a bit-mapped image into a caricature. 
Although described primarily with reference to the conversion of the facial images, the general 
techniques of FIG. 2 may be applied to converting other bit-mapped images into caricatures, 
such as converting the photograph of a house into a caricature of the house or converting the 
photograph of a person's body to a caricature. To this end, a separate set of image zones are 
defined, corresponding to standard features of homes. For example, an image zone 
corresponding to the doors and windows may be employed within a generally rectangular outline 
corresponding to the outline of the house. 

In many cases, the user may be satisfied with the caricature generated via the steps of 
FIG. 2 and will perform no further processing. In other cases, the user may wish to modify the 
caricature to adopt a predetermined artistic style or to merge the caricature with another 
caricature. To modify the caricature to adopt a selected artistic style, the steps of FIG. 3 are 
performed. To merge the caricature with another, the steps of FIG. 5 are performed. Referring 
first to FIG. 3, at step 140, the system inputs an artistic style (represented by item 141 of FIG. 
4) selected by the user for use in modifying the caricature. Exemplary artistic styles include 
styles associated with popular television programs, movies, comic books etc. The artistic styles 
may also be representative of animation techniques, such as cardboard cutout animation or pen 
and ink animation. One specific style is the Japanimation style associated with many Japanese 
cartoons. Each style is represented in vector form using the same types of geometric nodes as 
with the caricature, but sized and shaped so as to correspond with the particular artistic style. 
Hence, if the artistic style calls for characters with large bulbous eyes, nose and mouth, the nodes 
associated with the style are thereby pre-set to be wide apart (item 141 of FIG. 4). However, 
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whereas a particular caricature may be represented using a large number of nodes per image zone 
so as to fairly closely represent the original image, the style typically is defined using relatively 
few image nodes, so as to merely convey the general characteristics of cartoon characters of the 
artistic style rather than to correspond closely to a specific cartoon character. Each style is 
5 further represented by a set of drawing textures, color palettes and other rendering parameters 
for use in generating a caricature. Thus, for a style of animation wherein the characters appear 
as cardboard cutouts, the drawing textures cause the caricature to be rendered in a manner 
reminiscent of cardboard cut-outs. For a style of animation wherein the characters are drawn 
using thick, bold curvy lines, with vivid primary colors, the drawing textures cause the caricature 

10 to be rendered using bold, curvy lines and the color palette specifies primary colors. For a style 
of animation wherein the characters are drawn using thin, rough lines, with pastel colors, the 
drawing textures specify thin, rough lines and the color palette specifies pastel colors. As can 
be appreciated, numerous sets of textures, colors and other rendering features may be defined for 
the various artistic styles, each selected to convey the distinctive features of the associated style. 

1 5 The information specifying each artistic style is initially input by an artist or other operator of the 
system, who specifies the particular geometric nodes, color palettes, drawing textures and the like 
for each style. Within item 141 of FIG- 4, note that the computer system need not actually store 
the image of the cartoon shown therein. The image of the cartoon is provided in the figure 
merely to help to illustrate the idea of a general artistic style. 

20 Within step 142, the system then adjusts the nodes of the caricature generated based upon 

the user input photograph with the nodes of the selected artistic style. For example, the system 
extracts the image corresponding to the right eye of the user caricature (item 147 of FIG. 4) and 
adjusts the image based on the nodes of the artistic style (item 145 of FIG. 4) to generate a more 
bulbous right eye (item 149 of FIG. 4). A variety of techniques may be employed to perform the 

25 actual adjustment operation. In one example, the system first determines, for each node within 
the user caricature, a new location for the node by averaging the node locations from the user 
caricature with those of the artistic style based upon a predetermined biasing or weight factor 
which biases the average either toward the node location of the user caricature or to the node 
location of the artistic style. For example, the blending process may be fully biased towards the 

30 artistic style by setting the new node locations for the user caricature to be equal to the node 
locations of the artistic style. As a result, within a particular facial zone, the adjusted caricature 
image will more closely conform to the artistic style, ratiier than to the original user image. In 
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another example, the biasing is weighted heavily in favor of the user caricature such that the 
adjusted facial feature perhaps only slightly adopts the characteristics of the anistic style. 
Preferably, the biasing factors are input by the artist or other operator along with the other 
information defining the artistic style. To this end, the artist or other operator may subjectively 
5 evaluate the degree of distinctiveness of various features within the artistic style and assign very 
distinctive features a high biasing level and less distinctive features a lower biasing level. In this 
manner, when the adjustment procedure is performed, highly distinctive features of the artistic 
style are evoked within the adjusted image. Depending upon the programming of the system, the 
user may be permitted to adjust the weight factors as well. Also, as noted above, the artistic style 

10 may provide fewer nodes per facial feature than the user caricature provides. If so, the 
adjustment process thus far described is modified as needed to determine new locations for each 
node of the caricature based on the fewer node locations of the artistic style. 

Thus, following step 142, the system has generated a new set of node locations 
corresponding to the pre-determined image zones. Next, at step 144, the system automatically 

15 renders a new caricature based on the new set of node locations using the aforementioned artistic 
style textures, color palettes and the like to yield a final composite image (item 151 of FIG. 4). 
If the original caricature included colors, the new caricature can be rendered using the original 
colors modified using the color palette of the artistic style. As can be seen firom FIG. 4, the final 
image retains characteristics drawn from both the initial user photograph and the selected artistic 

20 style. The final image is displayed to the user and, if satisfactory, is employed by the user 
perhaps as part of the aforementioned computer message or perhaps as an Avatar for use in achat 
room or multi-game user computer game. If unacceptable, the user may select a different artistic 
style and repeat the process illustrated in FIG. 3 or, if permitted by the system, may attempt to 
selectively control the rendering process to achieve a more satisfactory result. Depending upon 

25 the programming of the system, the user caricature may be automatically rendered using many 
different artistic styles, with thumbnail sketches of each resulting caricature simultaneously 
displayed so that the user may select one. 

Thus, what has been described with reference to FIG. 3 is a technique for generating a 
new caricature based on the original caricature but rendered in a manner consistent with a 

30 selected artistic style. With reference to FIG. 5, techniques for merging the original caricature 
with another caricature will be described. At step 240, the system inputs a new caricature 
selected by the user for merging with the caricature previously generated by the system based on 
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the user photograph. The new caricature may be representative, for example, of a celebrity, 
historical figure or cartoon character. Such caricatures are preferably input into the system by 
' an artist or other operator. The artist or other operator inputs the specific geometric nodes 
corresponding to the facial features within each zone of the caricature. Preferably, a relatively 
large number of nodes are input to permit the caricature to closely resemble the particular 
celebrity, historical figure or cartoon character. Alternatively, the new caricature may be another 
caricature generated by the user, perhaps from a different input photograph. In either case, each 
caricature is stored within the system in vector form using the aforementioned geometrical nodes. 

Within step 242, the system then generates new node locations based on the nodes of the 
user caricature and the nodes of the selected caricature. This may be performed using the same 
method of FIG. 3 to average node locations from the user caricature with those of the other 
caricature based upon a predetermined biasing or weight factors. The system generates a final 
image of the merged caricature based on the new node locations for display to the user and, if 
satisfactory, the new caricature is employed by the user as part of the aforementioned computer 
message or perhaps as an Avatar for use in a chat room or multi-game user computer game. If 
unacceptable, the user may choose to select a different pre-storcd caricature for creating a merged 
character and repeat the process illustrated in FIG. 5 or, if permitted by the system, may attempt 
to selectively control the merging process to achieve a more satisfactory result. Depending upon 
the programming of the system, the user caricature may be automatically merged with many 
different pre-stored caricatures, with thumbnail sketches of each resulting merged caricature 
simultaneously displayed so that the user may select one. 

Thus, what has been described with reference to FIG. 5 is a technique for merging a 
caricature derived from a photograph with a pre-stored caricature. Alternatively, the general 
techniques of FIG- 5 may be employed to merge two separate caricatures provided by the user. 
Alternatively, the system may be programmed to enable the user to select two pre>stored 
caricatures for merging together. The system is not restricted to merging facial caricatures 
together. Rather, the technique may also be applied to merging other caricatures together, such 
as merging the caricature of a real house with that of a cartoon house. Also, the general 
technique of performing merging operations, on a zone-by-zone basis, may be applied to other 
merging techniques as well, including those not limited to the merging of caricatures together. 
The techniques of FIGS. 1 - 5 have been described with respect to flow charts. Each block 
within the flowcharts illustrated both a method step and an apparatus element for performing the 
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corresponding method step. Accordingly, FIGS. 1-5 also represent an image processing system. 
"The high-level features of the system are also represented within FIG. 6. 

Briefly, FIG. 6 illustrates functional components of a web site 300 performing the 
techniques of FIG. 1 - 5. The system includes an image selection unit 301 for inputting an image 
provided by a user via an input unit 302 or for allowing the user to select a pre-stored image from 
an image database 303. The system also includes a caricature generation unit 304 for converting 
the input image into a caricature, perhaps for transmission via a transmission unit 306 such as 
a modem over the Internet 308 or other computer network. Caricature generation unit 304 
converts a selected image into a caricature image using an image to geometrical node conversion 
unit 310, which extracts a set of geometrical nodes from the selected image, and a caricature 
rendering unit 312, which renders a caricature based on the extracted geometrical nodes. The 
nodes are extracted from the selected image on a zone-by-zone basis from set of predefined 
image zones, such as the eyes, nose and mouth. The zones are located within the image using 
an image zone locator unit 315. The geometrical nodes are stored in a node storage unit 313. 

The web site also includes an artistic style conversion unit 316 which may be used to 
modify the caricature to adopt an artistic style selected from an artistic style database 318. The 
system also includes a caricature merge unit 320 which may be used to merge a caricature 
generated by caricature generation unit 304 with a pre-stored caricature stored in vector-based 
format within node database 313. Specific operations performed by the various components 
illustrated in FIG. 6 are described above with reference to FIGS. 1 - 5. 

The description of the exemplary embodiments is provided to enable any person skilled 
in the art to make or use the invention. Various modifications to the embodiments will be readily 
apparent and generic principles defined herein may be applied to other embodiments. Thus, the 
invention is not intended to be limited to the specific embodiments shown and described herein 
but is to be accorded the widest scope consistent with the principles and novel features disclosed 
herein. 



19 



wo 01/59709 PCT/USOl/02160 

CLAIMS 

1. An interactive network web site comprising: 

an image selection unit for receiving a user selection of an image; and 
a caricature generation unit for automatically generating a caricature image 
representative of the selected image. 

2. The interactive network web site of claim 1 wherein the image selection unit includes 
an input unit for inputting an image provided by the user. 

3. The interactive network web site of claim 1 further including a selected image 
verification unit for determination whether the selected image meets a predetermined 
threshold of image integrity. 

4. The interactive network web site of claim 3 wherein the input image verification unit 
determines whether the selected image meets a predetermined threshold by analyzing one or 
more of the exposure, contrast and focus of the image. 

5. The interactive network web site of claim 1 wherein the image selection unit includes 
storage unit for storing images for selection by the user. 

6. The interactive network web site of claim 1 wherein the image selected by the user is a 
bit-mapped image and wherein the caricature generation unit generates a vector 
representation of the bit-mapped input image. 

7. The interactive network web site of claim 1 wherein the caricature generation unit 
locates a set of image zones v^ithin the selected image, separately converts portions of the 
selected image lying within each image zone into respective caricature image portions, and 
combines the caricature image portions to yield a resulting combined caricature image. 

8. The interactive network web site of claim 7 further including: 

an artistic style storage unit for storing information representative of a plurality of 
artistic styles 

an artistic style selection unit for receiving a user selection of one of the artistic styles; 

and 

an artistic style conversion unit for converting the caricature image based on the 
selected artistic style. 

9. The interactive network web site of claim 8 wherein the information representative of 
an artistic style includes information defining colors and textures. 

10. The interactive network web site of claim 1 further including a message generation 
unit for generating a message for transmission via the interactive network, for embedding 
the caricature image into the message, and for transmitting the message. 

11. The interactive network web site of claim 10 wherein the message generation unit 
receives a user selection of a message format from a set of pre-stored message formats and 
embeds the caricature image into the selected message format, 
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12. The interactive network web site of claim 1 1 wherein the pre-stored image formats 
include one or more formats directed to holiday greetings, birthday greetings, and reminder 
messages. 

13. The interactive network web site of claim 10 wherein the message generation unit 
inputs digitized speech provided by the user and embeds the digitized speech in the 
message. 

14. The interactive network web site of claim 1 further including a caricature merge unit 
for merging the caricature image with another caricature image to create a merged 
caricature image. 

15. The interactive network web site of claim 14 wherein the caricature merge unit locates 
a set of corresponding image zones within the caricature images, separately merges portions 
of the caricature images within each image zone into respective merged image portions, and 
combines the merged image portions to yield a resulting combined merged caricature 
image. 

16. The interactive network web site of claim 1 wherein the web site is connected to the 
Internet. 

17. A method performed by an interactive network web site comprising the steps of: 
receiving a user selection of an image; and 

automatically generating a caricature image representative of the selected image. 

18. The method of claim 17 wherein the step of receiving a user selection of an image 
includes the step of inputting an image provided by the user. 

19. The method of claim 17 further including the step of analyzing the selected image to 
determine whether the image meets a predetermined threshold of image integrity and 
rejecting the image if it falls below the threshold. 

20. The method of claim 19 wherein the step of analyzing the input image to determine 
whether the image meets a predetermined threshold includes the steps of analyzing one or 
more of the exposure, contrast and focus of the image. 

21. The method of claim 17 wherein the step of receiving a user selection of an image 
includes the step of displaying stored images for selection by the user. 

22. The method of claim 17 wherein the image selected by the user is a bit-mapped image 
and wherein the step of automatically generating a caricature image is performed to 
generate a 

vector representation of the bit-mapped input image. 

23. The method of claim 17 wherein the step of automatically generating a caricature 
image locates a set of image zones within the selected image, separately converts portions 
of the input image lying within each image zone into respective caricature image portions, 
and combines the caricature image pordons to yield a resulting combined caricature image. 
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24. The method of claim 17 further wherein the caricature image is a hne art image. 



25. The method of claim 17 further comprising the steps of: 
storing information representative of a plurality of artistic styles 

artistic style selection unit for receiving a user selection of one of the artistic styles; 

and 

converting the caricature image based on the selected artistic style. 

26. The method of claim 25 wherein the information representative of an artistic style 
includes information defining colors and textures. 

27. The method of claim 17 further including the steps of generating a message for 
transmission via the interactive network, for embedding the caricature image into the 
message, and for transmitting the message. 

28. The method of claim 27 wherein the step of generating a message includes the steps of 
receiving a user selection of a message format from a set of pre-stored message formats and 
embedding the caricature image into the selected message format. 

29. The method of claim 28 wherein the pre-stored image formats include one or more 
formats directed to holiday greetings, birthday greetings, and reminder messages. 

30. The method of claim 27 further including the step of inputting digitized speech 
provided by the user and embedding the digitized speech in the message. 

31. The method of claim 17 further including the step of merging the caricature image 
with another caricature image to create a merged caricature image. 

32. The method of claim 3 1 wherein the step of merging the caricature image with another 
caricature image locates a set of corresponding image zones within the caricature images, 
separately merges portions of the caricature images within each image zone into respective 
merged image portions, and combines the merged image portions to yield a resulting 
combined merged caricature image. 

33. A method, performed by a computing system, for converting an input image into a 
computer representation of a caricature image comprising the steps of: 

locating a set of image zones within the input image; and 

separately converting portions of the input image lying within each image zone into 
individual respective caricature representations; and 

combining the individual respective caricature representations to yield a combined 
caricature representation. 

34. The method of claim 33 wherein the input image is a facial image and wherein the set 
of image zones are facial image zones. 

35. The method of claim 33 wherein the facial image zones include the left eye, the right 
eye, the nose and the mouth. 
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36. The method of claim 33 wherein the step of locating a set of image zones within the 
input image includes the steps of: 

associating a set of discrete image zone templates with the input image, with each 
discrete image zone template corresponding to a respective one of the image zones; and 

converting portions of the input image corresponding to each discrete image zone 
template into a respective caricature image. 

37. The method of claim 36 wherein the set of discrete image zone templates are 
constrained with respect to an image template outline and wherein the step of associating 
the set of image zone templates with the input image includes the steps of: 

extracting the outline of the input image; 

adjusting a size and location of the image template outline to match the outline of the 
input image while adjusting sizes and locations of the discrete image zone templates in 
accordance with the adjusted size and location of the image template outline; and 

determining the final adjusted locations of the image zone templates with respect to 
the input image. 

38. The method of claim 33 wherein the input image is a bit-mapped image. 

39. The method of claim 38 wherein the step of separately converting portions of the input 
image lying within each image zone into individual respective caricature representations 
includes the step of, for each respective template zone, applying a corresponding 
predetermined bit-map to caricature conversion algorithm to the portion of the bit-map 
lying within the respective template zone. 

40. The method of claim 39 wherein at least some of the corresponding predetermined bit- 
map to caricature conversion algorithms differ from one another. 

41. The method of claim 40 wherein the input image is a facial image, the image zones of 
the facial image include the left eye,, the right eye, the nose and the mouth and wherein a 
first conversion algorithm is applied for the left and right eye template zones, a second 
conversion algorithm is applied to the nose template zone and a third conversion algorithm 
is applied to the mouth template zone. 

42. The method of claim 41 wherein the first conversion algorithm is adapted for 
converting a bit-mapped representation of an eye to a caricature representation of an eye, 
the second conversion algorithm is adapted for converting a bit>mapped representation of a 
nose to a caricature representation of a nose, and the third conversion algorithm is adapted 
for converting a bit-mapped representation of a mouth to a caricature representation of a 
mouth. 

43. The method of claim 33 wherein the caricature image is a line art image. 

44. A method, performed by a computing system, for rendering a caricature image 
comprising the steps of: 

inputting a caricature image representation; 
selecting an artistic style; and 

rendering an image based on the caricature image representation while applying the 
selected artistic style. 
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45. The method of claim 44 wherein the selected artistic style includes a specification of 
line textures and color palettes. 



46. The method of claim 45 wherein the step of rendering the image comprises the steps 
of: 

locating a set of image zones associated with the caricature image; 
identifying geometrical artistic style characteristics for use with each respective image 
zone; 

separately modifying portions of the caricature image representation within the image 
zones based on the geometrical artistic style characteristics identified for use with each 
respective image zone to yield separately modified image zone portions; 

combining the separately modified image zone portions. 

47. The method of claim 44 wherein the step of separately modifying portions of the 
caricature image within the image zone based on the geometrical artistic style 
characteristics identified for use with each respective image zone includes the step of, for 
each image zone, identifying a set of geometric features within the image zone and then 
adjusting the geometric features based on a corresponding set of geometric features of the 
artistic style for that zone. 

48. The method of claim 47 wherein different zones are assigned different weight factors 
and wherein the images lying within the image zones of the caricature image are adjusted 
based on the corresponding set of geometric features of the artistic style to a degree based 
on the weight factor. 

49. The method of claim 48 wherein the geometric features include predetermined 
geometric points and geometric tangents defining vectorized caricature images. 

50. The method of claim 47 wherein the images lying within the image zones of the 
caricature image are adjusted by averaging locations of the corresponding geometric points 
within the caricature image and the artistic style to yield a set of average locations of 
geometric features. 

51. The method of claim 44 wherein the artistic style is selected from a set of pre- 
determined artistic styles representative of different types of art. 

52. A method for combining computer representations of first and second images 
comprising the steps of: 

locating a set of corresponding imagejzones within the first and second images; 

separately modifying image zones of the first image based on characteristics of 
conresponding image zones of the second image to yield separately modified image zones; 
and 

combining the separately modified image zones to yield a modified image. 

53. The method of claim 52 wherein the first and second images are caricature images. 

54. The method of claim 53 wherein the first and second caricature images are facial 
images and wherein the set of image zones are facial image zones. 
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55. The method of claim 52 wherein the step of separately modifying image zones of the 
' first image based on characteristics of corresponding image zones of the second image 

includes the step of, for each image zone, identifying corresponding sets of geometric 
features within the first and second images and then adjusting the images lying within the 
image zone of the first image based on the corresponding sets of geometric features from 
the second image. 

56. The method of claim 55 wherein different zones are assigned different weight factors 
and wherein the images lying within the image zones of the first image are adjusted to a 
degree based on the weight factor. 

57. The method of claim 52 wherein the image zones contain caricature images and the 
sets of features include one or more of predetermined geometric points or geometric 
tangents within the caricature images. 

58. The method of claim 55 wherein the images lying within the image zones of the first 
image are adjusted by averaging locations of the corresponding geometric points within the 
first and second facial images to yield a set of average locations of geometric features, then 
modifying the images lying within the image zones of the first image based on the average 
locations of the geometric features of the second image. 

59. The method of claim 52 wherein in the first image is a bit-mapped image and the 
second image is a caricature image and wherein the method further includes the steps of : 

extracting the outline of the bit-mapped image; 
extracting the outline of the caricature image; 

conforming the outline of the caricature image to that of the bit mapped image to yield 
a modified caricature outline; and 

combining the modified caricature oudine with the modified image generated by 
combining the separately modified image zones to yield a combined modified image. 

60. The method of claim 52 wherein the first image is a bit-mapped photograph and the 
second image is a cartoon. 
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SEPARATELY CONVERT EACH IMAGE ZONE TO VECTOR-BASED REPRESENTATION USING 
DIFFERENT CONVERSION ALGORITHMS OPTIMIZED FOR DIFFERENT IMAGE ZONES WHICH 
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INPUT NEW CARICATURE SELECTED BY THE USER FOR MERGING WITH CARICATURE DERIVED 
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WITHIN EACH IMAGE ZONE. SEPARATELY MERGE USER CARICATURE WITH SELECTED NEW 
CARICATURE BY AVERAGING GEOMETRICAL NODE LOCATIONS OF USER CARICATURE WITH 
CORRESPONDING NODE LOCATIONS FROM NEW CARICATURE 
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